Surveys provide information that is important, and often critical, to decision making across a broad spectrum of endeavors. Among the most important and familiar of these is the U.S. Census, which strives to collect categorical information on every family in the United States and its territories. Although the goal of the census is to collect the information directly from the entire population, it has not been possible to do so. Rather, it is well established that the population of direct respondents does not include substantial and important portions of the population. And, it has become clear that the demography of the non-responding population is substantially different from that of the direct respondents. As a result, direct respondent census data contains important inaccuracies. The inaccuracies can be ameliorated to some extent by additional efforts to obtain the omitted information directly from the non-responding population. Such efforts, however, generally have been insufficiently successful to ensure that the census data represents the nation's population with sufficient accuracy for fairness in electoral redistricting and population-dependent allocation of federal resources. The census, accordingly, has applied statistical sampling methods to characterize, quantify, and statistically adjust the available data so that it more accurately portrays the true population.
Unlike the census, other polls and surveys do not seek responses from all members of an objective population. Rather, they seek to obtain a representative sample of the population, one that is as small as possible for a given degree of uncertainty when the sample data is applied to the population as a whole. Familiar methods that rely on statistical sampling of populations include, in the political area, for instance, both pre-election polling and exit polling of voters after they have cast their votes. Similar methods are crucial to market research surveys, which aim to characterize the response of a target “consumer” population to various products or services, and to specific aspects and designs thereof, among other things, from initiation of product development plans to decisions to discontinue a product.
For instance, product development cycles start with ideas, and idea generation is a critical first step in new product development. It belongs to the “fuzzy front end” of the development process, that has been recognized as a key leverage point. Dahan and Hauser (2001a) and Hauser et al. (2005). Many idea generation methods have been developed beginning in the 1950s. Brainstorming is probably the most popular method. Osborn (1957). Other idea generation methods include lateral thinking (De Bono (1970)), synectics (Prince (1970)), and six thinking hats (De Bono (1985)). More recently developed methods include electronic brainstorming, as described by Nunamaker et al. (1987), Gallupe et al. (1991), Gallupe et al. (1992), Dennis and Valacich (1993), and Valacich et al. (1994), ideation templates, as described by Goldenberg et al. (1999a), Goldenberg et al. (1999b), Goldenberg and Mazursky (2002), and incentives-based idea generation.
Successful idea generation processes typically produce a great many ideas. Often there are too many ideas to pursue them all. As a result, the ideas must be evaluated to select those on which to focus available resources. Several methods have been developed to systematize this process. One traditional approach is to ask experts to evaluate the ideas. Urban and Hauser (1993). However, experts are often costly, and their opinions often do not accurately reflect the needs and preferences of product users. Other methods suffer from similar disadvantages.
There is, therefore, a need to identify in a large group of ideas and other items those which merit further development effort. This is particularly so for ideas and other items generated by systematic idea generative processes. And while such methods are needed in many other areas, they are particularly needed for evaluating ideas and other items for developing consumer products.